Skip to content

Conversation

@idegtiarenko
Copy link
Contributor

@idegtiarenko idegtiarenko commented Feb 25, 2025

This adds list of target shards to operator description.

          {
            "operator": "LuceneSourceOperator[shards = [data:0], maxPageSize = 2340, remainingDocs = 999]", <----
            "status": {
              "processed_slices": 1,
              "processed_queries": [
                "*:*"
              ],
              "processed_shards": [
                "data:0"
              ],
              "processing_nanos": 112375,
              "slice_index": 0,
              "total_slices": 1,
              "pages_emitted": 1,
              "slice_min": 0,
              "slice_max": 0,
              "current": 2147483647,
              "rows_emitted": 1
            }
          }

@idegtiarenko idegtiarenko added >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL v9.1.0 labels Feb 25, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

StringBuilder sb = new StringBuilder();
sb.append(this.getClass().getSimpleName()).append("[");
sb.append("maxPageSize = ").append(maxPageSize);
sb.append("shards = ").append(sortedUnion(processedShards, sliceQueue.remainingShards()));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it is also worth including it in the status object as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about having it in the status only, instead of the toString? I imagine toString is a bit like an identity. So the decision would be whether we view this is part of the identify?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it should be. Each LuceneOperator is only used with list of shards it is originally initialized with

Copy link
Contributor

@craigtaverner craigtaverner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Had some minor comments, but nothing important.


public Iterable<LuceneSlice> getSlices() {
return slices;
public Collection<String> remainingShards() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we name this method remainingShardIds, since it returns strings, not shards?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that might be confused with org.elasticsearch.index.shard.ShardId. Probably this should be remainingShardIdentifiers() as it relies on shardContext().shardIdentifier()

@idegtiarenko idegtiarenko merged commit faa15eb into elastic:main Feb 26, 2025
17 checks passed
@idegtiarenko idegtiarenko deleted the add_profile_details branch February 26, 2025 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants